Validation of Artificial Intelligence to Support the Automatic Coding of Patient Adverse Drug Reaction Reports, Using Nationwide Pharmacovigilance Data.

INTRODUCTION: Adverse drug reaction reports are usually manually assessed by pharmacovigilance experts to detect safety signals associated with drugs. With the recent extension of reporting to patients and the emergence of mass media-related sanitary crises, adverse drug reaction reports currently frequently overwhelm pharmacovigilance networks. Artificial intelligence could help support the work of pharmacovigilance experts during such crises, by automatically coding reports, allowing them to prioritise or accelerate their manual assessment. After a previous study showing first results, we developed and compared state-of-the-art machine learning models using a larger nationwide dataset, aiming to automatically pre-code patients' adverse drug reaction reports. OBJECTIVES: We aimed to determine the best artificial intelligence model identifying adverse drug reactions and assessing seriousness in patients reports from the French national pharmacovigilance web portal. METHODS: Reports coded by 27 Pharmacovigilance Centres between March 2017 and December 2020 were selected (n = 11,633). For each report, the Portable Document Format form containing free-text information filled by the patient, and the corresponding encodings of adverse event symptoms (in Medical Dictionary for Regulatory Activities Preferred Terms) and seriousness were obtained. This encoding by experts was used as the reference to train and evaluate models, which contained input data processing and machine-learning natural language processing to learn and predict encodings. We developed and compared different approaches for data processing and classifiers. Performance was evaluated using receiver operating characteristic area under the curve (AUC), F-measure, sensitivity, specificity and positive predictive value. We used data from 26 Pharmacovigilance Centres for training and internal validation. External validation was performed using data from the remaining Pharmacovigilance Centres during the same period. RESULTS: Internal validation: for adverse drug reaction identification, Term Frequency-Inverse Document Frequency (TF-IDF) + Light Gradient Boosted Machine (LGBM) achieved an AUC of 0.97 and an F-measure of 0.80. The Cross-lingual Language Model (XLM) [transformer] obtained an AUC of 0.97 and an F-measure of 0.78. For seriousness assessment, FastText + LGBM achieved an AUC of 0.85 and an F-measure of 0.63. CamemBERT (transformer) + Light Gradient Boosted Machine obtained an AUC of 0.84 and an F-measure of 0.63. External validation for both adverse drug reaction identification and seriousness assessment tasks yielded consistent and robust results. CONCLUSIONS: Our artificial intelligence models showed promising performance to automatically code patient adverse drug reaction reports, with very similar results across approaches. Our system has been deployed by national health authorities in France since January 2021 to facilitate pharmacovigilance of COVID-19 vaccines. Further studies will be needed to validate the performance of the tool in real-life settings.

Genesis of an emergency public drug information website by the French Society of Pharmacology and Therapeutics during the COVID-19 pandemic.

On March 16, 2020, the French Society of Pharmacology and Therapeutics put online a national Question and Answer (Q&A) website, https://sfpt-fr.org/covid19 on the proper use of drugs during the COVID-19 pandemic. The working group 'Drugs and COVID-19' was composed of a scientific council, an editorial team, and experts in the field. The first questions were posted online during the first evening of home-confinement in France, March 17, 2020. Six weeks later, 140 Q&As have been posted. Questions on the controversial use of hydroxychloroquine and to a lesser extent concerning azithromycin have been the most consulted Q&As. Q&As have been consulted 226 014 times in 41 days. This large visibility was obtained through an early communication on Twitter, Facebook, traditional print, and web media. In addition, an early communication through the French Ministry of Health and the French National Agency for Medicines and Health Products Safety ANSM had a large impact in terms of daily number of views. There is a pressing need to sustain a public drug information service combining the expertise of scholarly pharmacology societies, pharmacovigilance network, and the Ministry of Health to quickly provide understandable, clear, expert answers to the general population's concerns regarding COVID-19 and drug use and to counter fake news.